Scalarizing Functions in Bayesian Multiobjective Optimization

Scalarizing functions have been widely used to convert a multiobjective optimization problem into a single objective optimization problem. However, their use in solving (computationally) expensive multi- and many-objective optimization problems in Bayesian multiobjective optimization is scarce. Scalarizing functions can play a crucial role on the quality and number of evaluations required when doing the optimization. In this article, we study and review 15 different scalarizing functions in the framework of Bayesian multiobjective optimization and build Gaussian process models (as surrogates, metamodels or emulators) on them. We use expected improvement as infill criterion (or acquisition function) to update the models. In particular, we compare different scalarizing functions and analyze their performance on several benchmark problems with different number of objectives to be optimized. The review and experiments on different functions provide useful insights when using and selecting a scalarizing function when using a Bayesian multiobjective optimization method

Treed Gaussian Process Regression for Solving Offline Data-Driven Continuous Multiobjective Optimization Problems

This is the final version. Available from MIT Press via the DOI in this recordFor offline data-driven multiobjective optimization problems (MOPs), no new data is available during the optimization process. Approximation models (or surrogates) are first built using the provided offline data and an optimizer, e.g. a multiobjective evolutionary algorithm, can then be utilized to find Pareto optimal solutions to the problem with surrogates as objective functions. In contrast to online data-driven MOPs, these surrogates cannot be updated with new data and, hence, the approximation accuracy cannot be improved by considering new data during the optimization process. Gaussian process regression (GPR) models are widely used as surrogates because of their ability to provide uncertainty information. However, building GPRs becomes computationally expensive when the size of the dataset is large. Using sparse GPRs reduces the computational cost of building the surrogates. However, sparse GPRs are not tailored to solve offline data-driven MOPs, where good accuracy of the surrogates is needed near Pareto optimal solutions. Treed GPR (TGPR-MO) surrogates for offline data-driven MOPs with continuous decision variables are proposed in this paper. The proposed surrogates first split the decision space into subregions using regression trees and build GPRs sequentially in regions close to Pareto optimal solutions in the decision space to accurately approximate tradeoffs between the objective functions. TGPR-MO surrogates are computationally inexpensive because GPRs are built only in a smaller region of the decision space utilizing a subset of the data. The TGPR-MO surrogates were tested on distance-based visualizable problems with various data sizes, sampling strategies, numbers of objective functions, and decision variables. Experimental results showed that the TGPR-MO surrogates are computationally cheaper and can handle datasets of large size. Furthermore, TGPR-MO surrogates produced solutions closer to Pareto optimal solutions compared to full GPRs and sparse GPRs.Academy of Finlan

Towards Better Integration of Surrogate Models and Optimizers

Surrogate-Assisted Evolutionary Algorithms (SAEAs) have been proven to be very effective in solving (synthetic and real-world) computationally expensive optimization problems with a limited number of function evaluations. The two main components of SAEAs are: the surrogate model and the evolutionary optimizer, both of which use parameters to control their respective behavior. These parameters are likely to interact closely, and hence the exploitation of any such relationships may lead to the design of an enhanced SAEA. In this chapter, as a first step, we focus on Kriging and the Efficient Global Optimization (EGO) framework. We discuss potentially profitable ways of a better integration of model and optimizer. Furthermore, we investigate in depth how different parameters of the model and the optimizer impact optimization results. In particular, we determine whether there are any interactions between these parameters, and how the problem characteristics impact optimization results. In the experimental study, we use the popular Black-Box Optimization Benchmarking (BBOB) testbed. Interestingly, the analysis finds no evidence for significant interactions between model and optimizer parameters, but independently their performance has a significant interaction with the objective function. Based on our results, we make recommendations on how best to configure EGO

Handling expensive multiobjective optimization problems with evolutionary algorithms

Multiobjective optimization problems (MOPs) with a large number of conflicting objectives are often encountered in industry. Moreover, these problem typically involve expensive evaluations (e.g. time consuming simulations or costly experiments), which pose an extra challenge in solving them. In this thesis, we first present a survey of different methods proposed in the literature to handle MOPs with expensive evaluations. We observed that most of the existing methods cannot be easily applied to problems with more than three objectives. Therefore, we propose a Kriging-assisted reference vector guided evolutionary algorithm (K-RVEA) for problems with at least three expensive objectives. The algorithm dynamically balances between convergence and diversity by using reference vectors and uncertainty information from the Kriging models. We demonstrate the practicality of K-RVEA with an air intake ventilation system in a tractor. The problem has three expensive objectives based on time consuming computational fluid dynamics simulations. We also emphasize the challenges of formulating a meaningful optimization problem reflecting the needs of the decision maker (DM) and connecting different pieces of simulation tools. Furthermore, we extend K-RVEA to handle constrained MOPs. We found out that infeasible solutions can play a vital role in the performance of the algorithm. In many real-world MOPs, the DM is usually interested in one or a small set of Pareto optimal solutions based on her/his preferences. Additionally, it has been noticed in practice that sometimes it is easier for the DM to identify non- preferable solutions instead of preferable ones. Therefore, we finally propose an interactive simple indicator-based evolutionary algorithm (I-SIBEA) to incorporate the DM’s preferences in the form of preferable and/or non-preferable solutions. Inspired by the involvement of the DM, we briefly introduce a version of K-RVEA to incorporate the DM’s preferences when using surrogates. By providing efficient algorithms and studies, this thesis will be helpful to practitioners in industry and increases their ability of solving complex real-world MOPs

A Feature Rich Distance-Based Many-Objective Visualisable Test Problem Generator

This is the author accepted manuscript. The final version is available from ACM via the DOI in this recordThe codebase for this paper is available at https://github.com/fieldsend/DBMOPP_generatorIn optimiser analysis and design it is informative to visualise how a search point/population moves through the design space over time. Visualisable distance-based many-objective optimisation problems have been developed whose design space is in two-dimensions with arbitrarily many objective dimensions. Previous work has shown how disconnected Pareto sets may be formed, how problems can be projected to and from arbitrarily many design dimensions, and how dominance resistant regions of design space may be defined. Most recently, a test suite has been proposed using distances to lines rather than points. However, active use of visualisable problems has been limited. This may be because the type of problem characteristics available has been relatively limited compared to many practical problems (and non-visualisable problem suites). Here we introduce the mechanisms required to embed several widely seen problem characteristics in the existing problem framework. These include variable density of solutions in objective space, landscape discontinuities, varying objective ranges, neutrality, and non-identical disconnected Pareto set regions. Furthermore, we provide an automatic problem generator (as opposed to hand-tuned problem definitions). The flexibility of the problem generator is demonstrated by analysing the performance of popular optimisers on a range of sampled instances.Engineering and Physical Sciences Research Council (EPSRC)Natural Environment Research Council (NERC

A survey on handling computationally expensive multiobjective optimization problems with evolutionary algorithms

Evolutionary algorithms are widely used for solving multiobjective optimization problems but are often criticized because of a large number of function evaluations needed. Approximations, especially function approximations, also referred to as surrogates or metamodels are commonly used in the literature to reduce the computation time. This paper presents a survey of 45 different recent algorithms proposed in the literature between 2008 and 2016 to handle computationally expensive multiobjective optimization problems. Several algorithms are discussed based on what kind of an approximation such as problem, function or fitness approximation they use. Most emphasis is given to function approximation-based algorithms. We also compare these algorithms based on different criteria such as metamodeling technique and evolutionary algorithm used, type and dimensions of the problem solved, handling constraints, training time and the type of evolution control. Furthermore, we identify and discuss some promising elements and major issues among algorithms in the literature related to using an approximation and numerical settings used. In addition, we discuss selecting an algorithm to solve a given computationally expensive multiobjective optimization problem based on the dimensions in both objective and decision spaces and the computation budget available.peerReviewe

An Interactive Simple Indicator-Based Evolutionary Algorithm (I-SIBEA) for Multiobjective Optimization Problems

This paper presents a new preference based interactive evolutionary algorithm (I-SIBEA) for solving multiobjective optimization problems using weighted hypervolume. Here the decision maker iteratively provides her/his preference information in the form of identifying preferred and/or non-preferred solutions from a set of nondominated solutions. This preference information provided by the decision maker is used to assign weights of the weighted hypervolume calculation to solutions in subsequent generations. In any generation, the weighted hypervolume is calculated and solutions are selected to the next generation based on their contribution to the weighted hypervolume. The algorithm is compared with a recently developed interactive evolutionary algorithm, W-Hype on some benchmark multiobjective optimization problems. The results show significant promise in the use of the I-SIBEA algorithm. In addition, the performance of the algorithm is demonstrated using a human decision maker to show its flexibility towards changes in the preference information. The I-SIBEA algorithm is found to flexibly exploit the preference information from the decision maker and generate solutions in the regions preferable to her/him.peerReviewe

A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem

Chugh T, Chakraborti N, Sindhya K, Jin Y. A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem. Materials and Manufacturing Processes. 2017;32(10):1172-1178.A new data-driven reference vector-guided evolutionary algorithm has been successfully implemented to construct surrogate models for various objectives pertinent to an industrial blast furnace. A total of eight objectives have been modeled using the operational data of the furnace using 12 process variables identified through a principal component analysis and optimized simultaneously. The capability of this algorithm to handle a large number of objectives, which has been lacking earlier, results in a more efficient setting of the operational parameters of the furnace, leading to a precisely optimized hot metal production process

A Data-Driven Surrogate-Assisted Evolutionary Algorithm Applied to a Many-Objective Blast Furnace Optimization Problem

A new data-driven reference vector-guided evolutionary algorithm has been successfully implemented to construct surrogate models for various objectives pertinent to an industrial blast furnace. A total of eight objectives have been modeled using the operational data of the furnace using 12 process variables identified through a principal component analysis and optimized simultaneously. The capability of this algorithm to handle a large number of objectives, which has been lacking earlier, results in a more efficient setting of the operational parameters of the furnace, leading to a precisely optimized hot metal production process.peerReviewe